Inferring Network Structure from Observation I: Binary Neural Networks

Inferring network structure from observed data is a useful procedure to study the relation between structure and function networks. For networks with observable dynamics but hidden structure, inference gives the best guess of the underlying connectivity that explains the observed data. For networks with known structure and observable dynamics, inference helps to separate parts of the network that directly contribute to the dynamics and function and those that don’t. Both aspects of network inference may find their uses in neuroscience in the future: inferring connectivities of hundreds of interconnected neurons recorded concurrently by microscopic imaging, or deconstructing the functional connections in a heuristically trained artificial neural network for example. In this study, we give theoretical answers to two important questions of network structure inference. First, using binary neural networks as an example of single-step, discrete-time network, we study and characterize the constraint on individual node’s dynamics for the network inference problem to be solvable using convex optimization. Second, as L1 regularization is often used to solve large-scale sparse problems the case for most real networks, we derive and verify a closely form way of calculating the corresponding regularization parameter given either the prior knowledge or the estimated value for sparsity. This study was first motivated by On the Convexity of Latent Social Network Inference by S.A. Myers and J. Leskovec 2010 in which they used convex optimization to infer structures of information diffusion networks. The binary neural network was modified from the network analyzed in Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks by N. Bertschinger and T. Natschläger in 2004. Instead of individual nodes having binary values of -1 and 1, we use 0 and 1. The theoretical approach to find the optimal regularization parameter was inspired by Bayesian Regularization and Pruning Using a Laplace Prior by P.M. Williams in 1995, in which a heuristic choice of regularization parameter was given to cases with no prior sparsity knowledge.